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The use of the loudest observed event to generate statistical statements about rate and strength has become 
standard in searches for gravitational waves from compact binaries and pulsars. The Bayesian formulation of 
the method is generalized in this paper to allow for uncertainties both in the background estimate and in the 
properties of the population being constrained. The method is also extended to allow rate interval construction. 
Finally, it is shown how to combine the results from multiple experiments and a comparison is drawn between the 
upper limit obtained in a single search and the upper limit obtained by combining the results of two experiments 
each of half the original duration. To illustrate this, we look at an example case, motivated by the search for 
gravitational waves from binary inspiral. 



I. INTRODUCTION 

In daily life, we often estimate the birth rate, the rate of au- 
tomobile fatalities, or the rate of hurricanes in the Gulf. In 
these cases, it is reasonably easy to determine when one event 
has occurred and so the best estimate is usually taken to be the 
number of events divided by the observation time. As physi- 
cists and astronomers, we know this is a good estimator of 
the rate of an underlying Poisson process. In these cases, the 
ability to identify events with high confidence is central to the 
correctness of the rate estimate. 

We can carry this method over to more complicated obser- 
vational situations by allowing for false positives in our iden- 
tification of events. Experiments are usually designed so that 
the rate of real (foreground) events is higher than the rate of 
false positive (background) events. Hence a good estimate of 
the rate is obtained by counting the number of events per unit 
time, and making a small correction to allow for the false pos- 
itives. This is the typical experimental method of estimating 
the rate. 

In both physics and astronomy, it is common to search for 
very rare events in large data sets and we rely heavily on sta- 
tistical methods to interpret these searches. In this paper, we 
discuss the problem of estimating the rate of these rare events. 
When real events are very rare or very weak, it is important 
to revisit the reasoning that underlies the standard approaches 
to estimating rates (and indeed other parameters). Here, we 
explore the effects of incorporating information about quality 
of observed events into the estimate of event rate. One mea- 
sure of quality might be the signal to noise of the events; the 
louder an event, the more likely it is to be signal. Of course, 
more complicated measures are also possible. We simply re- 
quire a rank ordering such that a larger quality implies the 
event is less likely to be background. 

A popular method of incorporating quality information is 
to fix a threshold, prior to looking at the data. The thresh- 
old is often chosen to give an acceptable rate of background 
events in some qualitative sense. Then, the upper limit is de- 
termined by counting the number of events per unit time above 



the chosen threshold and making a correction which allows for 
the background. Central to this method is the prescription by 
which the final list of events are identified. 

There are many alternative criteria that might be used to 
determine the sample of events in an experiment. We con- 
sider using the loudest event to estimate the rate. This method 
was first introduced in gravitational-wave searches during the 
analysis of data from a prototype instrument (ill; the method 
was used to determine an upper limit on the rate of bi- 
nary neutron star mergers in the Galaxy. Since then, the 
method has been used in a number of searches for gravita- 
tional waves More details of this method 
of determining an upper limit are available in Ht]. Related 
methods have been discussed in the context of particle physics 
experiments by Cousins and YelUn ifioll . 

In Sec. ini we present a general formulation of the loudest 
event statistic llllll]. We adopt the Bayesian approach which 
gives a posterior distribution over physical parameters based 
on the loudest event observed in an experiment. To provide a 
concrete example, in Sec.|III]we specialize to the case of a sin- 
gle unknown rate amplitude multiplying a known distribution 
of events. Confidence intervals based on the loudest event 
posterior are discussed in Sec. |IV] The approach we take is 
sometimes called a highest posterior density interval ifTHl . It 
provides a unified approach giving an upper limit for a loud- 
est event that is due to noise with high probability and a con- 
fidence interval (bounded away from zero) when the loudest 
event is foreground with high probability. In real experiments 
there are many systematic uncertainties; we discuss marginal- 
ization over uncertainties in Sec. [V] Finally, we explain how 
to combine the results from multiple experiments using the 
loudest event method and show that the resulting upper limit is 
independent of the order of the experiments. This discussion 
also leads naturally to an investigation of the effect on an up- 
per limit if a single search is split into two parts. In Sec. I VIII 
we make some general comments on the results obtained in 
this paper In Appendix |A] we consider the application of a 
Feldman-Cousins unified approach to obtaining a frequentist 
upper limit using the loudest observed events. A compari- 
son between the loudest event method and an event counting 
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method of obtaining an upper limit is given in AppendixiBlfor 
a toy problem. 

Notation: We use the following notation throughout this 
paper The loudest event statistic variable is denoted x. The 
experimentally measured values of a quantity are denoted by 
a circumflex accent; e.g., the experimentally measured value 
of the loudest event statistic is given by x. Probability distri- 
butions and other quantities related to an experimental back- 
ground appear with a subscripted 0; e.g., Pq {x) is the back- 
ground probability distribution for the loudest event statistic. 
The symbol B is used to describe the collective information 
about the experimental background in conditional probabil- 
ity distributions that are contingent on this information; thus 
P{x\B) is the distribution of the loudest event statistic in an 
experiment that includes a background. 

II. GENERAL FORMULATION OF LOUDEST EVENT 
STATISTIC 

Consider a search of experimental data for a rare Poisson 
process. The output of this search is a set of candidate events 
which have survived all cuts applied during the analysis. At 
first, suppose that all the events are foreground events. As- 
sume that these events can be ranked according to a single 
parameter x, such as a signal-to-noise ratio, in such a way that 
the probability that the search will detect an event increases 
with increasing x. For simplicity we will call this parameter 
the loudness parameter and we will say that the event with the 
largest value of x is the loudest event. If the mean number 
of events expected during the course of the experiment with 
the ranking statistic value above x is given by vlx), then the 
probability of observing no events above a given value of x is 

= e~''(^). (1) 



and po{x) = dP{x)/dx. Notice that the probability distribu- 
tion contains two factors: an exponential decay that is deter- 
mined by the (foreground) rate of events and a shape factor 
comprising two terms. 



However, if the experiment can produce background events, 
then the probability that there are no events, either foreground 
or background, louder than x is 

P{x\B) = Po(a;)e-''(^) (2) 

where we have used B to indicate that the probability depends 
on the background and the factor Po{x) is the probability of 
obtaining zero background events louder than x. 

The mean number of events expected during the course of 
the experiment, I'lx), depends on the duration of the experi- 
ment, the rate of events, and the ability of the experiment to 
detect events that occur The sensitivity of the search is en- 
coded in the efficiency which is the probability that an event 
will have a loudness value greater than or equal to x. The effi- 
ciency depends on x as well as a set of parameters, collectively 
denoted by 9, that determine the detectability of a source. For 
example, 9 may include such things as the sky position, ori- 
entation, distance, etc., of an astrophysical source. We write 
the efficiency as e{x, t, 9). (The sensitivity of the experiment 
may change with time; hence the explicit dependence on t in 
the efficiency.) The rate of events depends on the parameters 
that describe the source population as well as on physical 
parameters, collectively denoted by that we are interested 
in measuring or constraining by means of the experiment. We 
write the rate of events as R{9,fi). With this factorization, 
the mean number of events expected can be expressed as 

v{x,fj,)^j^ dt j dee{x,t,9)R{9,n), (3) 

where T is the total observation time. 

We can substitute our expression for the rate (O into Eq. (|2]i 
to obtain the probability that there are zero events in the data 
with a loudness statistic value greater than x as 



(4) 



(5) 



rate. To do this, we calculate a Bayesian posterior distribution 
for these parameters, /x, given the observations. This distribu- 
tion is denoted p (^i I a; , B), where x is the value of the observed 
loudest event, and it is derived using Bayes' law: 



P(.t|/i,S) = Po(a;)exp dt j d9 e{x,t,9)R{9, n) 



Furthermore, the probability of the loudest event occurring between x and a; + da; is given by p{x\fi, B) dx where 



p{x\^Ji,B) - —P(x\fi,B) 



1 - 



Pojx) 







''dt fd9^^^^^R{9,f.) 

dx 



After performing an experiment, we are interested in ob- 
taining a distribution for the model parameters that govern the 
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where is the prior probability distribution on the model 
parameters. In many circumstances, the parameters fj, may be 
further divided into a set of particular interest /Xj and others 
of less interest /Xjj. By integrating Eq. ^ over the unwanted 
parameters /Xjj, one obtains the posterior distribution 

p{Hl\x,B) = jr- — . (7) 

In Sec. |V] we consider this procedure of marginalization over 
unwanted, or nuisance, parameters in more detail. 

To bound the parameters of interest at a given confidence 
level a, one integrates Eq. ^ over some region ^{fJ-i) such 
that 



/ 



dfJ-p{pi\x,B). 



(8) 



In general, the difficult part is selecting the region es- 
pecially in more than one dimension. There are several ways 
to do this: for example, one could marginalize over all but 
one of the parameters thus reducing the problem to a one- 
dimensional integral; or select the smallest volume that 
gives the required probability. This is sometimes called a 
highest posterior density interval ifTTll . In Sec. IIVI we inves- 
tigate the properties of this type of rate interval based on the 
loudest event method. 



III. UPPER LIMIT ON UNKNOWN RATE AMPLITUDE 

We have obtained the general expression for the posterior 
probability distribution of the parameters fi governing an as- 
trophysical model based on an observed loudest event. In 
practice, the details of obtaining either a rate upper limit or 
a confidence interval on the model parameters will depend 
upon the details of the astrophysical model and its dependence 
upon the variables fi. In this section, we simplify to the situ- 
ation where the rate is dependent upon a single parameter /i, 
an overall unknown Poisson mean number of events, so that 



T 



(9) 



where T is the observation time and f{0) is the distribution 
of events as a function of 0. 

We can use this form of the rate to simplify the general ex- 
pression for the posterior To begin, we introduce the quantity 

e{x) = ^ dt J dee{x,t,e)f{e) (lo) 

which can be regarded as an averaged detection efficiency: 
the probability that a foreground event will have a loudness 
parameter greater than x. Then, the mean number of events 
with ranking statistic above x is iy{x) ~ /ie(.T), and (at least 
in principle) e(x) is known. The posterior distribution is de- 
termined by substituting Eqs. (|9]l into OJ and using Eqs. (|5]l, 
©, and ([Tol l to obtain 



pin\e,A) 



p{fi)po{x){l + fieA)e-''' 
J dfip{fi)po{x) (1 + ^JLek)e-^'^ 



(11) 



where the function A(x) is given by 
/ -1 de{x 



e{x) dx J \Pi^{x) 



(12) 



and a hat over a function indicates evaluation at x. The quan- 
tity A(a;) is a measure of the relative probability of detecting 
a single event with loudness parameter x versus such an event 
occurring due to the experimental background; in particular 
A ^ in the limit that the loudest event is definitely from the 
background and A ^ cxd in the limit that the loudest event is 
definitely from the foreground. Note that if the possibility that 
the loudest event could be from a background is ignored, the 
posterior distribution, p(/i|e) oc e e^'"^ is peaked away from 
zero and vanishes as /i ^ 0; that is, the posterior distribution 
will be inconsistent with zero foreground events. 

For the rest of the paper we take a uniform prior except in 
Section [VT] in which we consider using the posterior from a 
first experiment as a prior for a second experiment. It should 
be noted that only power law priors on the rate, including the 
uniform prior, do not introduce a timescale into the problem. 
Power laws with powers greater than -1 are needed to avoid a 
required low-rate cutoff, which again would introduce a natu- 
ral timescale. 

Let us evaluate the upper limit making use of a uniform 
prior. 



p(/.t) = const. 



(13) 



While this distribution is not normalizable, we can introduce 
a cutoff at large ^ (well above the expected number of events 
during the given experiment) in order to render it normaliz- 
able. Physically, this is a reasonable choice of prior if there is 
no information available about the expected value of ji. Fur- 
thermore, the posterior distribution is insensitive to the value 
of the cutoff provided it is sufficiently large. For the uniform 
prior, the posterior distribution in Eq. ( fTTT i evaluates to 



p{^l\e,^) 



l + A 



(1 + nek)e-^'' 



(14) 



It is straightforward to show that the distribution in Eq. ( fTTT i 
will be peaked away from zero if and only if A > 1; the mode 
of the distribution is 



Mpcak 



A < 1 

(A-1)/Ae A>1. 



(15) 



If A > 1 then one might take this as an indication of a non- 
zero rate. The extent to which this is true is explored in 
SecHV] 

We integrate Eq. ( fT4l i to obtain an upper limit at confidence 
level a by solving 



dn'pi^i'\e,A) 



l + A 



-He{x) 



(16) 
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FIG. 1: The posterior probability density function p(/i|e, A) on the 
Poisson mean /i, assuming a uniform prior, and a fixed value e of the 
efficiency evaluated at the loudest event x. The three curves corre- 
spond to three different values of A: a) A = (solid line), the loudest 
event is definitely background and the distribution is exponential; b) 
A = 1 (dashed line), the transitional case where the the distribution 
peaks at zero but the derivative vanishes there; c) A ^ oo (dotted 
line), the loudest event is definitely from the foreground, the distri- 
bution is peaked away from zero. 



for yu. It has been shown in id] that setting the background 
to zero yields a conservative rate limit. In the Bayesian anal- 
ysis, however, this yields a posterior probability distribution 
function which is peaked away from zero, and goes to zero 
at zero rate. This is clearly seen in Fig. [T] which shows the 
posterior distribution for three values of A including A ^ oo. 
This is not surprising as we have neglected the background, 
in which case the existence of a loudest event implies a non- 
zero rate. Although this does not invalidate the upper limit, 
it does mean that the posterior would not serve as a suitable 
prior for a future experiment, as it is inconsistent with a zero 
rate. Nevertheless, it is still possible to obtain the upper limit 
as 



M90% _ 3.890 
" Te{x) 



T 



(17) 



Similarly, the no-foreground limit can be obtained by taking 
A = 0. In this case, the 90% confidence limit tends to 



/^9o% _ 2.303 
T ~ Te{x) ' 

Finally, we can consider the transitional case A = 1: 
/^90% 3.272 



(18) 



T 



Te{x)- 



(19) 



The posterior distribution for the Poisson mean /i for these 
three possibilities is shown in Fig.[T] 



FIG. 2: The graph shows the behavior of the lower and upper bound- 
aries of the interval, /ii and /i2 respectively, as a function of A. They 
are plotted for three different values of the confidence level a of 80%, 
90% and 95%. The peak ^poak (solid line) approaches zero as A ap- 
proaches one. As A ^ 0, ^2 agrees with the no foreground upper 
limit treated above. 



IV. CONFIDENCE INTERVAL ON UNKNOWN RATE 
AMPLITUDE 

In Sec. [nil we derived the upper limit on the Poisson mean 
/i based on the loudest event. However, in the case where the 
value of A is large (likely to be foreground), one might pre- 
fer to obtain a rate interval rather than an upper limit. For 
a uniform prior, the mode /ipcak of the posterior distribution 
for the Poisson mean, given in Eq. ( fTST i, is non-zero when- 
ever A > 1. Furthermore, in this case, /ipcak asymptotes to 
1/e for large values of A as one might expect. How signifi- 
cant an indicator of a non-zero rate is having the peak of rate 
distribution be non-zero? In order to examine this idea more 
precisely, we describe a method of constructing a rate inter- 
val using the loudest event statistic which provides a unified 
approach similar to Feldman and Cousins II12I1 . 

At some confidence level a, an interval is given by [/ii , 112] 
such that 



p(/i|e, A) d/i = a. 



(20) 



A supplementary condition is required to select a unique in- 
terval: we identify the interval which minimizes \fi2 — /^il 
and contains the mode of the distribution (or zero for A < 1). 
This condition clearly results in /ii =0 for small values of 
A, i.e. when the loudest event was Hkely to have arisen from 
the background, the rate interval on the process we wish to 
constrain includes zero rate. 

For the uniform prior, the dependence of /ii, fi2 and /ipcak 
on A are shown in Fig. [2] For A < 1, /ipcak = and conse- 
quently /ii = 0, as expected. However, for a significant range 
of A > 1, even though the rate distribution is peaked away 
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from zero, fii = indicating that (at the given confidence) 
the rate interval still includes zero. 

We can determine the precise value of A at which fii be- 
comes non-zero. For fixed A and e, Eq. ( |20] | gives /i2 implic- 
itly as a function of /ii . The minimal interval condition is then 
just 



dm 



= 0. 



(21) 



Substituting /ii = into Eqs. ( |20] | and ( |2TI ). we obtain two 
equations which depend on ^2 and A. As an example, con- 
sider a 90% confidence interval. In this case, /ii becomes 
non-zero, and the interval is bounded away from the origin, at 
value of A ~ 11.56. This corresponds to /i2 — 3.807/e(i;). 
This result is in good agreement with the values obtained nu- 
merically in Fig. |2] 

It is interesting to note that the 90% confidence interval still 
includes zero for a wide range of A that give posterior distri- 
butions peaked away from zero. Figure |3]provides a concrete 
example of the posterior when A = 10; the 90% confidence 
interval still includes zero. 



V. MARGINALIZATION OVER UNCERTAINTIES 

The expected mean number of detected events, v{x, fi) in 
Eq. (|3]l, is dependent upon the frequency of events and their 
amplitude distribution as well as the sensitivity of the search 
which is performed. In many cases, neither of these quantities 
will be precisely known. For example, the efficiency of an 
experiment is often measured via Monte-Carlo methods and 
therefore suffers from uncertainties due to the finite number 
of trials. If we expand our understanding of the parameters 
fi to further parametrize the uncertainties that can arise in the 
underlying models and in measurements of efficiency, it is nat- 
ural to marginalize over these uncertainties before computing 
an upper limit or rate interval. Just as the marginalization over 
uninteresting physical parameters [given in Eq. O] requires a 
prior distribution to be specified, the same is true of the un- 
certainties. This prior distribution would typically reflect the 
systematic and statistical errors estimate for the experiment. 



A. Marginalization over uncertainties in e 

As a particular example, consider the problem of the un- 
known rate amplitude presented in Sec. |III]and assume there 
is some uncertainty associated with the value of e = £(•■£)■ 
Typically, one might choose the prior to be a normal distribu- 
tion of the variate e peaked around the estimate value of e. It 
is, however, unphysical for the rate to be zero, so the distribu- 
tion would need to be truncated. A more natural choice is a 
log-normal distribution, for which the logarithm of e would be 
normally distributed, thereby guaranteeing that e is positive. 

Here, we choose to make use of the 7-distribution, pri- 
marily because it can be analytically integrated. The 7- 
distribution is similar in shape (for small standard deviation) 
to both the Gaussian and log-normal distributions and in ad- 
dition takes only non-negative values. The 7-distribution is 
given by 



g(fc-l)g-e/e 

6"=r(fc) 



(22) 



where T{k) is the Gamma function. The mean ise = k9 while 
the standard deviation is = k^^^O. Therefore, the fractional 
standard deviation, a^/e = k~^/^ tends to zero in the limit as 
k ^ 00, whereby we expect to recover the unmarginalized 
results. Note that the 7-distribution is a distribution over the 
domain e e [0, 00) while the efficiency actually takes values 
only between and 1 . If fc is large then the 7-distribution is 
sharply peaked about its mean value and we can ignore this 
issue. 

The marginalized distribution is calculated by integrating 
over e. 



p{^i\k,e,A) 



dep{e;k,0)p{^j,\e,A) 



(23) 



e=e/k 



where we set the value of the parameter 9 so that the mean of 
the 7-distribution equals the observed efficiency e, and where 
/c is a measure of the fractional uncertainty in the value of e. 
Making use of the distribution ( fTTT ) for the Poisson mean pa- 
rameter and the expression for the 7-distribution given above, 
we obtain the marginalized distribution for integer values of k 



p{fi\k,e,A) = 



(1 + A) 



1 

(i + iii/k) 



fe+i 



^leA{l + l/k) 

(1 + fii/k)''+^ 



(24) 



In the limit that k — > 00, we recover the previous distribution 
for yu as expected. 

In order to examine the effect of marginalization, in Fig. [3] 
we plot the unmarginalized posterior distribution for A = 10 
along with three distributions obtained by marginalizing over 
different size systematic errors or uncertainties. These distri- 



butions are obtained from ( l24b with values of fc = 100, 16 and 
4 corresponding to errors of 10%, 25% and 50% respectively. 
As the systematic error increases, the posterior distribution 
for the Poisson mean parameter gets broader; the value of the 
probability density function increases for large values of the 
Poisson mean parameter This causes an increase in the upper 
limit. Without taking into account any uncertainties, the 90% 
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/i p Systematic Error {%) 



FIG. 3: The posterior probability density function on the Poisson 
mean parameter jj, for different sizes of systematic error, where e is 
the efficiency evaluated at the loudest event x. The curves were gen- 
erated assuming a uniform prior and using A = 10. The solid line 
corresponds to the unmarginalized probability density function. The 
dot-dashed line gives the distribution marginalized over a 10% sys- 
tematic uncertainty (equivalently k — 100 for the 7-distribution). 
With this level of uncertainty, the marginalized distribution is barely 
changed from the original. The dotted and dashed lines show the 
posterior for 25% (k = 16) and 50% (k = 4) systematic errors. As 
the systematic error increases the distribution broadens and conse- 
quently the upper limit increases. 



confidence upper limit is 3.796/e(i). For 10% systematic er- 
ror, this increases only slightly to 3.850/e(.T) while for 25% 
and 50% this increases further to 4.147 /e{x) and 5.434/e(a;) 
respectively. In Figure |4] we plot the upper limit as a func- 
tion of the systematic error for four different values of A. The 
results are qualitatively similar to what was seen before — 
marginalizing over uncertainties will increase the upper limit 
and the larger the errors, the larger the effect. 



B. Marginalization over uncertainties in A 

In many cases, there will also be uncertainties in the precise 
value of A = A(a;). These can be marginalized over in the 
same way as described above. Since the A dependence of the 
distribution (fTTT l is straightforward, this can be done explicitly. 
For concreteness, let us take a uniform prior, in which case 
the posterior distribution is given by Eq. ST4i . Then, given a 
probability distribution p{A), the marginalized distribution is 



FIG. 4: The 90% confidence upper limit versus the size of the sys- 



p(/i|e) = / dAp{A)p{fi\e,A) 



(25) 



In this case, the above integral is straightforward. Specifically, 
let us define 



J dApiA) - 



A 



A)- 



(26) 



tematic error which is marginalized over (equivalent to k~ ' in the 
7-distribution discussed in the text). The limit is plotted for four dif- 
ferent values of A: 0, 0.1, 1, 10. In all cases, the upper limit increases 
with larger systematic error. 



Then, the posterior distribution following marginalization 
over A is given by 



p{m = n{^-o+f^me-''' 



(27) 



where ^ contains all of the dependence of the posterior on the 
marginalized background. 

Suppose that A is distributed with expectation value A and 
variance a\. Then, to leading order, 



A 



1 + A (1 + A)3 



(28) 



From this, we notice two things. First, even if the fractional 
uncertainties in A are of order unity, when A ^ 1 or A <C 
1, the second term is small compared to the first and can be 
ignored. Second, marginalizing over A only serves to decrease 
the value of ^ relative to the unmarginalized case. This is 
equivalent to reducing the likelihood that the loudest event 
is foreground and consequently will reduce the upper limit. 
Therefore, it is possible to neglect the marginalization of A as 
this is a conservative thing to do. 



VI. COMBINING RESULTS FROM MULTIPLE 
EXPERIMENTS 

When performing a series of experiments, there is a very 
natural way to combine the results in a Bayesian manner. As 
discussed above, the calculation of a Bayesian upper limit re- 
quires the specification of a prior probability distribution for 
the rate fi. When a previous experiment has been performed, 
it is natural to use the posterior from the first experiment as 
the prior for the second. It is straightforward to show that the 
results are independent of the order of the experiments. (This 
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does not depend upon the loudest event, rather it is a general 
Bayesian result.) Begin by recalling that 



J dnp{fi)p{xi\n)' 



(29) 



For the second search, simply use p{fi\xi) as the prior to ob- 
tain the posterior distribution on fi given the observations in 
both the first and second experiments: 

J d^lp(^l)p(Xl\fl)p(x2\^l) 

This is clearly symmetric in xi and X2- It is straightforward to 
see that marginalization over nuisance parameters (see SeclVTi 
preserves this symmetry. 

Let us consider this in more detail. If the first search was 
performed using a uniform prior, the posterior is given by 
Eq. ( fTTT i with a loudest event value xi observed in the first 
search. Furthermore, in the event that the loudest event is 
most likely background, one expects Ai <C 1. Then, we can 
conservatively rewrite the posterior as 



(M|ei,Ai) -eiAie-^^'i(i-^i) (31) 



^^conservative 



where ii = ei{xi), Ai = Ai(xi), and we have made use of 
the fact that 



1 + MeiAi < e^^'i^i 



(32) 



It is straightforward to show that the rate limit at a given confi- 
dence level a inferred using this posterior is necessarily larger 
than that obtained using the original distribution. In this sense, 
the alternative distribution is conservative and the distribution 
has been cast as an exponential. 

Therefore, in the second search, it is natural to use an expo- 
nential prior. 



p{p,) = Ke 



(33) 



To obtain the posterior distribution obtained when the expo- 
nential prior is used, it is beneficial to re-define A{x) as 

A.(.)=f^^V^V' (34) 



where 



e^{x) dx J \Pn{x) 



ei^{x) = e{x) + K 



(35) 



includes the exponential scale constant from the prior distri- 
bution. Then, the posterior distribution is given by 



with = ei^{x) and A„ = Ai^{x). As before, the posterior 
distribution is peaked away from zero if A^ > 1. In addition, 
the distribution is identical to that obtained using a uniform 
prior, only now the search efficiency is effectively e{x) + k. 
A. Splitting a search 



Next, let us consider the effect of taking a single search and 
splitting it into two halves, which can be combined to produce 
an upper limit in the manner described above. Naively, it ap- 
pears that splitting the search will give a lower rate limit, since 
we will be using a quieter loudest event for half the search. If 
this were the case, then it would seem that splitting the search 
into ever shorter searches would lower the upper limit indef- 
initely. As we shall see, the result is not so clear cut, and it 
depends critically upon the foreground and background distri- 
butions e{x) and Po{x). 

Consider an experiment performed for some given time T, 
and assume that both the foreground and background rates are 
constant over time. We would then like to compare the (ex- 
pected) upper limit from the full search to that obtained by 
splitting the data in two parts of length Ti and T2 and cal- 
culating a combined upper limit from the two searches. Let 
us assume, without loss of generality, that the loudest event 
overall in the search occurs in the first half of the search with 
a statistic value of xi, and the loudest event in the second half 
of the search has a statistic value X2- Then, we can calculate 
the upper limit from the search (taking it as a single entity) 
and from the split search. 

The posterior for the single search is given by 



p{n\xi,B) 



p{fi) [1 + fie{xi)A{xi)] exp{~fie{xi)} 



J dfip{fi) [1 + fj.e{xi)A{xi)] exp{-ne{xi)} 

(37) 

while for the spht search, the likelihood for each part is pro- 
portional to 



p{xi\p.,B) 



■A, 



(38) 



(36) 



where i = 1,2 label the two parts of the search, — ei{xi), 
Ai = Ai{xi) and r]i = Ti/T is the fraction of the total obser- 
vation time that is contained in the each interval; 771+772 = 1- 
Then the combined posterior distribution for the split search 
is 
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1 


+ jU77ieiAi 




1 


+ ^1112(2^2 


cxp{-/_t[7?iei + 112^2]} 




1 +/x7?ieiAi 




1 + fi7]2i2M 


exp{-/i[77iei + 71262]} 



(39) 
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FIG. 5: The ratio of /iaingie to /Xgpiit as a function of A2 for sev- 
eral values of e2/ei- The data presented uses a uniform prior on /i, 
Tji — |, e{x) — ei{x) and Ai(a::) — A2{x). The figure was gener- 
ated for A{xi) = 0.5Ai(xi) — 1. In general, there is only a weak 
dependence on this value; the curves steepen a little for smaller value 
of A(a;i), but look qualitatively similar. Note also that for most sen- 
sible choices of amplitude statistic x, one expects A2 < Ai. The plot 
is extended to A2 = 10 for completeness. 



We can now compare the posterior distributions for the sin- 
gle and split search. In general, the efficiencies ei{x) and 
the likelihoods Ai{x) could be different for the two parts. 
Nevertheless, it follows directly from Eq. (fTOl i that e{x) = 
T]iei{x) + r]2e2{x). Therefore, in the split search, the expo- 
nential decay term is at least as large as for the single search, 
with equality only if X2 = xi. This tends to make the upper 
limit obtained in the split search smaller than that of the single 
search. In contrast, the polynomial prefactor is always more 
significant for the split search (i.e. it grows more steeply with 
/i). This tends to make the upper limit larger. So splitting the 
search will lead to a larger limit when £2 = ei and A2 = Ai. 
Meanwhile if £2 ^ ei and A2 ^ Ai, the split search will give 
a numerically smaller limit. 

Assuming a uniform prior on /i, we compare the 90%- 
confidence upper limits on the Poisson mean obtained from 
a single search with the limit obtained from a split search with 
7?i = 7?2 = 1/2. The results are shown in Fig. |5] for sev- 
eral choices of £2/61 under the assumptions that e{x) = ei{x) 
and Ai{x) = A2(x). While not the most general case, these 
assumptions are reasonable in the context of an experiment 
with the same apparatus and background noise sources. In the 
limit A2 0, we find /Xgingio > /ispiit as expected. As A2 
increases, the second event is less likely to be background and 



so the rate limit from the split search can become bigger than 
that obtained in the single search. While this makes intuitive 
sense, the result depends on the particular observed outcomes 
of the experiment. 

When Ai ^ 1 and A2 ■€i 1, the posterior distribution for 
the single search can be approximated conservatively as 



p{fi\xi,B) ~ e(ii)[l - A{xi)]e-^'^^''>^^-^^^'^^ 
while the posterior for the split search becomes 

p(m|.ti,X2,S) ^c(xi,i2)e-^^(*^'*=) 

where 

c(xi,X2) = ei77i(l - Ai) + e27?2(l - A2). 



(40) 



(41) 



(42) 



If we further assume that both the foreground and background 
are Poisson distributed over the entire search, then ei{x) ~ 
e{x) and riiAi{x) = A{x). Within the context of these as- 
sumptions, it is then easy to write down the upper limit for 
each distribution. In particular. 



Msinglc 



ln(l - a) 



(43) 



e(xi)[l-A(xi)] 

for the single search; for the split search 

-ln(l - a) 
Mspiit = . ' (44) 

C(Xi,X2j 

Hence, the single search will give a smaller upper limit if 

e{xi) 



A2(i2) > 



1 



e{x2) 



(45) 



Once again, the comparison between the single and split 
search is sensitive to the precise nature of the foreground, 
background and observed results. 



VII. DISCUSSION 

The loudest event statistic is just one method of taking 
account of the quality of an event in the interpretation of a 
search. In this paper, we have presented further exploration of 
the method including the discussion of marginalization over 
uncertainties in the input model. The Bayesian approach al- 
lows simple accounting of these uncertainties by integrating 
them out. 

We also showed how the method could be used to deter- 
mine a rate interval. Once again, this is not the most power- 
ful method of determining an interval (in the sense that using 
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more than one event would lead to a more strongly peaked 
distribution and, consequently, a narrower interval). Never- 
theless, the approach shows that a rate interval arises when 
the likelihood that the event is signal becomes large enough. 

Finally, we presented a discussion of combining the results 
from multiple searches to determine a single upper limit. It 
was shown that the limit obtained by combining two searches 
of equal duration is, in general, different to the limit obtained 
by performing a single search of equivalent duration. What 
conclusion to draw from this is unclear since the notion of 
better depends on the true value of the rate being explored. 

Even though physicists have a deep appreciation for proba- 
bilistic phenomena in nature, it is often tempting to talk about 
better upper limits by using one method or another This is, of 
course, a flawed approach. In fact, it is the experiment that one 
should choose not the statistical method. Nevertheless, some 
experiments may be more powerful than others. For example, 
it would be ill-conceived to use the loudest event method to 
determine a rate interval in an experiment which is likely (in 
the sense of prior probability) to generate more than one loud 
event that could be considered to arise from the phenomenon 
of interest. Indeed, these considerations lead back to an ex- 
periment more like the standard threshold approach. 
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APPENDIX A: FREQUENTIST APPROACH TO THE 
LOUDEST EVENT STATISTIC 

This paper has explored the loudest event statistic from the 
Bayesian point of view. In this appendix we consider an al- 
ternative approach: the construction of frequentist confidence 
intervals using the loudest event statistic. The construction 
of frequentist upper limits is almost trivial (see Isj]). More 
interesting is the application of the method of Feldman and 
Cousins 111211 for a unified approach to constructing confidence 
intervals. We restrict attention here to the case in which only 
the unknown rate amplitude /i is to be bounded. 

We briefly summarize the Neyman approach to construct- 
ing confidence belts: For each fixed value of ^, an interval 
[xi, X2\ is constructed such that the probability of observing 
a loudest event x in this interval is equal to the desired confi- 
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FIG. 6: The a = 90% confidence belt constructed using the proce- 
dure of Feldman and Cousins for the example described in the text. 
For each value of ^, the probability of obtaining a loudest event value 
of X in the interval given by the solid line bounded by open circles 
is 90%. The solid diamond shows the point in this interval where 
R{x) is the greatest. For an observed loudest event value x, the in- 
terval on /i is the intersection of the vertical line x = x with this belt. 
Note that for small values of x an upper limit on /i is obtained while 
for larger values of x the interval on ji is bounded away from zero. 
The division between an upper limit and an inteval that excludes zero 
occurs at a value of 2:90% for which -Po(a^go%) = 90%. 



dence level a: 

P{x2\ix) - P[xi\ix) = a (Al) 

where 

P{x\^i) ^ PQ{:x)e-^'<'■'■\ (A2) 

The collection of such intervals then defines a confidence belt; 
for any observed value of the loudest event x, the belt covers 
a range of values of /i, [/ii, /i2], which is the desired interval 
on the rate amplitude parameter. In order to construct the con- 
fidence belt, a supplementary condition is needed in order to 
uniquely define the interval. For example, to obtain a confi- 
dence belt that always yields upper limits, choose X2 — 00. 
Then the belt is defined as the interval [xi, 00) where xi sat- 
isfies 

l-P(xi|/i) = a (A3) 

for each value of /i. Then, given an observed loudest event 
value X, the rate amplitude interval is [0, fi2] where 

1 - P{x\fi2) = a (A4) 
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ln(l -g) -\nPo{x) 



(A5) 



As mentioned in |8ji|, this procedure has the pathology that if 
Po{x) < 1 — a then the interval on fi is empty. 

The unified approach of Feldman and Cousins provides 
a different supplementary condition for defining the inter- 
val [a;i,a;2] for fixed /i. In the Feldman and Cousins ap- 
proach, the interval is constructed using a function R{x) so 
that R{x) > Rmin for x G [xi, .'E2] and R{x) < i?min for x 
outside the interval where i?,„i„ = min{i?(.Ti), R{x2)}- The 
function R{x) is chosen to be the likelihood ratio 



R{x) 



p{x\^J■pcl^k) 

[1 + /ie(x)A(a;)]e-'"(^) A(.t) < 1 



(A6) 



— A(a:) > 1 



where 



pix\n) 



dP{x\^i) 
dx 



A(a;)ei/A(aO- 



= Po(2:)[l+Me(^)A(x)]e-^^(^) (A8) 



and /ipcak is given by Eq. ( fTSl ). 

As an illustration, we compute the confidence belt accord- 
ing to the Feldman and Cousins procedure for the follow- 
ing example: We take €{x) = (a^min/a^)'^ and Pq{x) = 

1 — exp(.Tmin — x) with I'lnin = 5; Only values X > Xmin 

are realizable. The a = 90% confidence belt is shown in 
Fig. |6] Notice that there is a well-defined inteval on /i for 
any observed loudest event value x possible, that is, the Feld- 
man and Cousins procedure produces confidence belts that are 
free of the pathology described above when upper limit con- 
fidence belts are constructed. Furthermore, the interval on ji 
is a upper limit for small loudest event values x < XgQ%, but 
becomes an interval which excludes zero for x > xgQ% where 
Po{x90%) = 90%. 

It is interesting to consider the behavior of the confidence 
intervals for large values of x. In this regime, A(x) ^ 1, 
Po{x) ~ 1 so R{x) ~ /ie(a;)e-^"(^)+i and P{x\fi) ~ 



g-Me(3:j fjjg confidence belt at fixed ii is given by the in- 
terval [a;i,a;2] that satisfies R{xi) = R{x2) and P(x2|/i) — 
P{xi\^) = a. Therefore, xi and X2 satisfy the coupled 
equations fj,e{xi)e-''<''^^ = fj,e{x2)e->"'^''^^ and e^^'^^^) - 
^-M^i) ^ ^_ poj. Q, = 90%^ ^Jie{xl) = 3.932 and 
/ie(x2) = 0.08381. Consequently, the 90% confidence 
rate amplitude interval [/ii , 112] for large x will be given by 
Ml = 0.08381/e(i) and ^2 = 3.932/e(i). Notice the ratio 
~ 46.91 is fixed: the fractional uncertainty in the value 
of ji does not improve as x increases. This demonstrates that 
for experiments in which events in the low -background region 
are expected, the loudest event statistic will not give strong 
constraints on the event rate. As emphasized in the introduc- 
tion, the loudest event statistic is best suited to problems in 
which the anticipated event rate is very low and foreground 
and background events are expected to have comparable am- 
plitudes. 
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no. 7: a) The upper limit as a function of the observed loudest 
event. The solid line shows the value of the upper limit as a func- 
tion of X. The dotted and dashed lines are given by 2.303/e(a;) and 
3.890/e(i). We see that the upper limit transitions smoothly from 
one to the other. At low values of x, the loudest event is very much 
consistent with the background, A ~ and the upper limit is close to 
the dotted line. For larger values of x the loudest event is more con- 
sistent with foreground, A 00, and the upper limit is more con- 
sistent with the dashed line, b) The probability distribution for the 
loudest event assuming that it is drawn from the background distri- 
bution, po(x). Multiplying the upper limit curve by this distribution 
and integrating over x gives the expected value of the upper limit if 
the loudest event is from the background. 



APPENDIX B: COMPARISON WITH FIXED THRESHOLDS 



Let us compare the loudest event statistic against a fixed 
threshold approach. The loudest event prescription can be ap- 
plied to any form of background, provided the required quan- 
tities in Eq. ( fT2b can be measured or estimated. In many ex- 
periments, one might expect the background events above a 
statistic value x to be Poisson distributed, with mean vq{x) 
where is a non-increasing function of x. Then, it follows 
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FIG. 8: Figure showing the expected upper limit as a function of 
the fixed threshold. The dashed line shows the upper limit obtained 
when ignoring the background, while the dotted line includes the 
background contribution. For large values of the threshold where 
the expected background is small, both limits approach 2.303/e(2;*) 
as expected. For low values of a;*, there is a good chance of many 
events above threshold which leads to a worse upper limit. The bal- 
ance occurs at around a threshold value of x* = 8.3. For reference, 
we also plot a horizontal line showing the expected upper limit from 
the loudest event. Interestingly, the loudest event will, on average, 
outperform the fixed threshold for any value of the threshold. 



directly that 



Po{x) 
Po{x)/Po{x) 



dx 
dvoix) 



-1/0(2:) 



dx 



(Bl) 



We 



work with an example where the mean i^o{x) = 
g(8 -2:')/2 ^jj(j(jjg foj-ggj-omj(j jg (jjg(j-jl3^(g(j g^l^,^ — (8/a;)'^. 
These choices are natural in the context of a search for gravi- 
tational waves from coalescing binaries where the background 
is distributed with two degrees of freedom, while the fore- 
ground is uniformly distributed in volume, and the signal 



strength (and hence statistic value x) are inversely propor- 
tional to the distance lfl3ll . The normalizations of these func- 
tions are chosen for simplicity so that i'o{8) = e(8) = 1. The 
main feature of these distributions, however, is simply that 
they are both decreasing functions of x, and that the back- 
ground decreases more rapidly than the foreground. The value 
of the upper limit as a function of the actual loudest event is 
shown in Fig. |7^. The upper limit transitions smoothly from 
the zero foreground limit (at low values of x) to zero back- 
ground limit (at large values of x). Figure |7j5 shows the dis- 
tribution po{x). This corresponds to the expected distribution 
of for the loudest event if it is due to the background. Then, 
by multiplying the upper limit by the expected distribution for 
the loudest event and integrating, we obtain the expected up- 
per limit. In this example it is 2.64. 

For comparison, the upper limit for a fixed threshold is pre- 
sented in Figure |8] When calculating the upper limit for a 
fixed threshold, one simply counts the number of events n 
above the chosen threshold x* and obtains a limit 



T 



e{x*)T 



(B2) 



where Fjn) is a known function for each integer n (see, for 
example, |[l4ll for more details). In particular, when zero 
events are observed above the threshold, F{0) = 2.303. 
When performing a fixed threshold search, it is possible to 
take into account the expected background and, much as for 
the loudest event, neglecting to do so will lead to a conserva- 
tive result. In Fig. [8] we show the expected upper limit as a 
function of the threshold. 

Clearly, in this example, the loudest event statistic is prefer- 
able to a fixed threshold, as it will provide a better expected 
upper limit value than the fixed threshold for any value of the 
threshold (with or without the background). We note that this 
result is specific to the details of the example under considera- 
tion; the key feature is that the background rate is a very steep 
function of x. Indeed, in [[s]], the same example was consid- 
ered, but with an expected background of unity at x = 4.5 
rather than x = 8, leading to a small range of values where 
the fixed threshold does beat the loudest event. However, as 
emphasized in that paper the attraction of the loudest event is 
that it is unnecessary to fix a threshold ahead of performing 
the search — the search itself determines the threshold. 
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